Partial information decomposition as a unified approach to the specification of neural goal functions

In many neural systems anatomical motifs are present repeatedly, but despite their structural similarity they can serve very different tasks. A prime example for such a motif is the canonical microcircuit of six-layered neo-cortex, which is repeated across cortical areas, and is involved in a number of different tasks (e.g. sensory, cognitive, or motor tasks). This observation has spawned interest in finding a common underlying principle, a ‘goal function’, of information processing implemented in this structure. By definition such a goal function, if universal, cannot be cast in processing-domain specific language (e.g. ‘edge filtering’, ‘working memory’). Thus, to formulate such a principle, we have to use a domain-independent framework. Information theory offers such a framework. However, while the classical framework of information theory focuses on the relation between one input and one output (Shannon’s mutual information), we argue that neural information processing crucially depends on the combination of multiple inputs to create the output of a processor. To account for this, we use a very recent extension of Shannon Information theory, called partial information decomposition (PID). PID allows to quantify the information that several inputs provide individually (unique information), redundantly (shared information) or only jointly (synergistic information) about the output. First, we review the framework of PID. Then we apply it to reevaluate and analyze several earlier proposals of information theoretic neural goal functions (predictive coding, infomax and coherent infomax, efficient coding). We find that PID allows to compare these goal functions in a common framework, and also provides a versatile approach to design new goal functions from first principles. Building on this, we design and analyze a novel goal function, called ‘coding with synergy’, which builds on combining external input and prior knowledge in a synergistic manner. We suggest that this novel goal function may be highly useful in neural information processing

Strain dependence of the acoustic properties of amorphous metals below 1K: Evidence for the interaction between tunneling states

We have conducted a thorough study of the acoustic properties between 10^-4 and 1 Kelvin for the amorphous metal Zr_x Cu_1-x (x=0.3 and x=0.4), by measuring the relative change of sound velocity dv/v and internal friction Q^-1 as a function of temperature and also of the applied strain, in both superconducting and normal state. We have found that when plotted versus the ratio of strain energy to thermal energy, all measurements display the same behavior: a crossover from a linear regime of ``independent'' tunneling systems at very low strains and/or high enough temperatures to a nonlinear regime where dv/v and Q^-1 depend on applied strain and the tunneling systems cannot be considered as independent.Comment: 4 pages, 4 figures (submitted to PRL

Contextually Guided Unsupervised Learning Using Local Multivariate Binary Processors

We consider the role of contextual guidance in learning and processing within multi-stream neural networks. Earlier work (Kay & Phillips, 1994, 1996; Phillips et al., 1995) showed how the goals of feature discovery and associative learning could be fused within a single objective, and made precise using information theory, in such a way that local binary processors could extract a single feature that is coherent across streams. In this paper we consider multi-unit local processors with multivariate binary outputs that enable a greater number of coherent features to be extracted. Using the Ising model, we define a class of information-theoretic objective functions and also local approximations, and derive the learning rules in both cases. These rules have similarities to, and differences from, the celebrated BCM rule. Local and global versions of Infomax appear as by-products of the general approach, as well as multivariate versions of Coherent Infomax. Focusing on the more biologically plausible local rules, we describe some computational experiments designed to investigate specific properties of the processors. The main conclusions are : 1. The local methodology introduced in the paper has the required functionality. 2. Different units within the multi-unit processors learned to respond to different aspects of their receptive fields. 3. The units within each processor generally produced a distributed code in which the outputs were correlated, and which was robust to damage; in the special case where the number of units available was only just sufficient to transmit the relevant information, a form of competitive learning was produced. 4. The contextual connections enabled the information correlated across streams to be extracted, and, by improving feature detection with weak or noisy inputs, they played a useful role in short-term processing and in improving generalization. 5. The methodology allows the statistical associations between distributed self-organizing population codes to be learned

Is weak temperature dependence of electron dephasing possible?

The first-principle theory of electron dephasing by disorder-induced two state fluctuators is developed. There exist two mechanisms of dephasing. First, dephasing occurs due to direct transitions between the defect levels caused by inelastic electron-defect scattering. The second mechanism is due to violation of the time reversal symmetry caused by time-dependent fluctuations of the scattering potential. These fluctuations originate from an interaction between the dynamic defects and conduction electrons forming a thermal bath. The first contribution to the dephasing rate saturates as temperature decreases. The second contribution does not saturate, although its temperature dependence is rather weak, $\propto T^{1/3}$. The quantitative estimates based on the experimental data show that these mechanisms considered can explain the weak temperature dependence of the dephasing rate in some temperature interval. However, below some temperature dependent on the model of dynamic defects the dephasing rate tends rapidly to zero. The relation to earlier studies of the dephasing caused by the dynamical defects is discussed.Comment: 14 pages, 6 figures, submitted to PR

Low-temperature anomalous specific heat without tunneling modes: a simulation for a-Si with voids

Using empirical potential molecular dynamics we compute dynamical matrix eigenvalues and eigenvectors for a 4096 atom model of amorphous silicon and a set of models with voids of different size based on it. This information is then employed to study the localization properties of the low-energy vibrational states, calculate the specific heat C(T) and examine the low-temperature properties of our models usually attributed to the presence of tunneling states in amorphous silicon. The results of our calculations for C(T) and "excess specific heat bulge" in the C(T)/T^3 vs. T graph for voidless a-Si appear to be in good agreement with experiment; moreover our investigation shows that the presence of localized low-energy excitations in the vibrational spectrum of our models with voids strongly manifests itself as a sharp peak in C(T)/T^3 dependence at T < 3K. To our knowledge this is the first numerical simulation that provides adequate agreement with experiment for the very low-temperature properties of specific heat in disordered systems within the limits of harmonic approximation.Comment: 5 pages with 2 ps figures, submitted to PR

Magnetic field effect on the dielectric constant of glasses: Evidence of disorder within tunneling barriers

The magnetic field dependence of the low frequency dielectric constant $e_r$(H) of a structural glass a - SiO2 + xCyHz was studied from 400 mK to 50 mK and for H up to 3T. Measurement of both the real and the imaginary parts of $e_r$ is used to eliminate the difficult question of keeping constant the temperature of the sample while increasing H: a non-zero $e_r$(H) dependence is reported in the same range as that one very recently reported on multicomponent glasses. In addition to the recently proposed explanation based on interactions, the reported $e_r$(H) is interpreted quantitatively as a consequence of the disorder lying within the nanometric barriers of the elementary tunneling systems of the glass.Comment: latex Bcorrige1.tex, 5 files, 4 figures, 7 pages [SPEC-S02/009

Rings and rigidity transitions in network glasses

Three elastic phases of covalent networks, (I) floppy, (II) isostatically rigid and (III) stressed-rigid have now been identified in glasses at specific degrees of cross-linking (or chemical composition) both in theory and experiments. Here we use size-increasing cluster combinatorics and constraint counting algorithms to study analytically possible consequences of self-organization. In the presence of small rings that can be locally I, II or III, we obtain two transitions instead of the previously reported single percolative transition at the mean coordination number $\bar r=2.4$, one from a floppy to an isostatic rigid phase, and a second one from an isostatic to a stressed rigid phase. The width of the intermediate phase $~ \bar r$ and the order of the phase transitions depend on the nature of medium range order (relative ring fractions). We compare the results to the Group IV chalcogenides, such as Ge-Se and Si-Se, for which evidence of an intermediate phase has been obtained, and for which estimates of ring fractions can be made from structures of high T crystalline phases.Comment: 29 pages, revtex, 7 eps figure

Noise Kernel and Stress Energy Bi-Tensor of Quantum Fields in Hot Flat Space and Gaussian Approximation in the Optical Schwarzschild Metric

Continuing our investigation of the regularization of the noise kernel in curved spacetimes [N. G. Phillips and B. L. Hu, Phys. Rev. D {\bf 63}, 104001 (2001)] we adopt the modified point separation scheme for the class of optical spacetimes using the Gaussian approximation for the Green functions a la Bekenstein-Parker-Page. In the first example we derive the regularized noise kernel for a thermal field in flat space. It is useful for black hole nucleation considerations. In the second example of an optical Schwarzschild spacetime we obtain a finite expression for the noise kernel at the horizon and recover the hot flat space result at infinity. Knowledge of the noise kernel is essential for studying issues related to black hole horizon fluctuations and Hawking radiation backreaction. We show that the Gaussian approximated Green function which works surprisingly well for the stress tensor at the Schwarzschild horizon produces significant error in the noise kernel there. We identify the failure as occurring at the fourth covariant derivative order.Comment: 21 pages, RevTeX

Energy landscape, two-level systems and entropy barriers in Lennard-Jones clusters

We develop an efficient numerical algorithm for the identification of a large number of saddle points of the potential energy function of Lennard- Jones clusters. Knowledge of the saddle points allows us to find many thousand adjacent minima of clusters containing up to 80 argon atoms and to locate many pairs of minima with the right characteristics to form two-level systems (TLS). The true TLS are singled out by calculating the ground-state tunneling splitting. The entropic contribution to all barriers is evaluated and discussed.Comment: 4 pages, RevTex, 2 PostScript figure

Frequency behavior of Raman coupling coefficient in glasses

Low-frequency Raman coupling coefficient of 11 different glasses is evaluated. It is found that the coupling coefficient demonstrates a universal linear frequency behavior near the boson peak maximum and a superlinear behavior at very low frequencies. The last observation suggests vanishing of the coupling coefficient when frequency tends to zero. The results are discussed in terms of the vibration wavefunction that combines features of localized and extended modes.Comment: 8 pages, 9 figure